Continuous time dynamics of the Thermal Minority Game 
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We study the continuous time dynamics of the Thermal Minority Game. We hnd that the 
dynamical equations of the model reduce to a set of stochastic differential equations for an interacting 
disordered system with non-trivial random diffusion. This is the simplest microscopic description 
which accounts for all the features of the system. Within this framework, we study the phase 
structure of the model and find that its macroscopic properties strongly depend on the initial 
conditions. 
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Many of the current challenges for statistical physics 
have their origins in problems in biology ffl and eco- 
nomics In particular, the application of ideas and 
techniques of the statistical mechanics of disordered sys- 
tems can prove useful in the study of systems of adaptive 
and competitive agents, which are relevant, for example, 
to the microscopic modeling of financial markets; and, 
conversely, such problems can raise new issues for statis- 
tical physics. One of these systems is the minority game 
(MG) |H||], a simple model based on Arthur's "El Farol" 
bar problem || which describes the behaviour of a group 
of competing heterogeneous agents subject to the eco- 
nomic law of supply and demand. Despite its simplicity, 
the MG is very non-trivial, and although much progress 
has been made in the qualitative and quantitative 
p0| , pl| understanding of its features, a full analytic solu- 
tion of the MG is still missing. 

The main hurdles in the way of an analytical study of 
the MG in its original formulation were its non-locality in 
time due to the dependence on the game history, its dis- 
crete kinematics and dynamics, and the "best-strategy" 
rule (see, however, |n|). The hrst of these obstacles was 
overcome in ]l2"| , where it was shown numerically that the 
macroscopic behaviour of the MG was unchanged if the 
real history was replaced by a random one. This allowed 
the study of a simpler stochastic Markovian problem in- 
stead of the original deterministic non-Mar kovian one. 

In (l3) a natural continuous generalization of the MG 
was presented. The "information" to which the agents re- 
acted was taken as an external input to the system and 
it was shown that all the macroscopic features of the MG 
were preserved, as long as the external information was 
ergodic in time, the simplest choice being just noise. To 
handle the problem of the 'best-strategy' rule, the Ther- 
mal Minority Game (TMG) was introduced, in which a 
certain degree of stochasticity in the choice of the strate- 
gies by the agents was allowed, controlled by a parameter 
T, the "temperature" , the limits T = and T = 00 cor- 
responding to the original deterministic MG and the case 
of completely random strategy choices, respectively. The 
TMG displayed extra non-trivial structure as a function 
of T, notably that in the region where the MG performs 
worse than random, the system can be made to perform 
better than random by allowing a certain degree of indi- 



vidual stochastic error. 

In the present paper we carefully study the continu- 
ous time limit of the TMG, in order to obtain the sim- 
plest microscopic description which accounts for all the 
macroscopic features of the system, and as a further step 
towards an analytical solution of the model. We confirm 
that the external information 'integrates-out' providing 
simply an effective coupling between agents. We also 
show the crucial dependence of the macroscopic prop- 
erties of the model on the initial conditions. We find 
that the microscopic equations of the TMG reduce to a 
set of disordered stochastic differential equations with a 
non-trivial random diffusion matrix, and study the phase 
structure of the model in the temperature/dimension 
plane. 

The setup of the TMG is as follows p|. The system 
consists of N agents playing the game. At each time 
step t, each agent reacts to a common piece of "infor- 
mation" J(t), by making a "bid" bi(t) (i — 1,...,N), 
The information, defined as a unit-length vector in M. D , 
is taken to be a random noise, ^-correlated in time and 
uniformly distributed on the unit sphere |14|. The bid 
b(t) is defined to be a real number, which can be in- 
terpreted as placing an order in a market, of size \b(t)\ 
and positive/negative meaning buy/sell. Bids are pre- 
scribed by "strategies" : maps from information to bid, 
MP — > R. For simplicity the strategy space T of the 
model is restricted to the subspace of homogeneous lin- 
ear mappings. A strategy R is defined as a vector in M D , 
subject to the constraint ||i?|| = \J~D, and the prescribed 
bid is just the scalar product R ■ I(t). Each agent has 
S strategies, drawn randomly and independently from T 
(with uniform distribution) remaining fixed throughout 
the game. In what follows we will restrict for simplicity 
to the case of two strategies per agent S — 2, the gener- 
alization to S > 2 being straightforward. At time step 
t each agent i chooses one of his/her strategies R*{t) 
to play with. The "total bid" (or "excess demand") is 
then A(t) = = Ej^W ' fy)- The agents 

keep track of the potential success of the strategies by 
assigning points to them, which are updated according 
to P(R,t+ 1) = P{R,t) - A(t)b(R)/N, where P(R,t) 
represents the points of strategy R at time t. 
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In the original formulation of the MG the agents played 
in a deterministic fashion using their 'best' strategies, 
the ones with the highest number of points. In the 
TMG the natural generalization to non-deterministic be- 
haviour is allowed. At time step t, each agent i chooses 
R*(t) randomly from his/her {Rj,R 2 } with probabil- 
ities {wj (t), wf(t)}. The probabilities nf{t) are func- 
tions of the points parameterized by a temperature T, 
defined so as to interpolate between the MG case at 
T = 0, all the way up to the totally random case 
it] = tt| = 1/2 at T = oo. The qualitative behaviour 
of the system does not depend on the specific functional 
form of the 7rf (t). In [H| the probabilities were defined 
as nl' 2 (t) oc exp [f3P{R]' 2 ,t)} (with nj (i) + tt? (t) = 1 and 
[3 = 1/T), while an alternative convenient form is given 
by tt^ (t) oc exp [±/3,Zj(t)], where Zi(t) = sgn(pi(t)), with 
Pi(t) ee [P(Rj,t) - P(R 2 ,t)}/2. The consequential dif- 
ference between these two definitions will be discussed 
below. 

The set of unconstrained degrees of freedom of the 
TMG is given by the difference Pi(t) of the points of 
the two strategies of each agent. The choice of strategies 
can then be defined by R*(t) = hi + & sgn [si(t) + ft(t)], 
where s,(t) ee nj(t) - n 2 (t), uj t = (Rj + R 2 ) /2, £ ee 
(Rj — R 2 ) 1 2, and n(i) is a stochastic random variable 
uniformly distributed between —1 and 1 and indepen- 
dently distributed in time. The equations for the point 
differences then read, 



p i (t + l)=p i {t)-a{t)-I{t)£ i -I(t), 



(1) 



where a(t) = T,i^*( t )/ N - E q s - (0)> together with the 
random processes for I{t) and R*(t), define the dynam- 
ics of the TMG. 

The average of the total bid A(t) over time and 
quenched disorder is zero, so the first relevant macro- 
scopic observable of the TMG is its normalized stan- 
dard deviation a (or "volatility") a 2 = N- 1 (A 2 {t)}, 
where the overline means disorder average, and (•) ee 
limt-xx, t f* 0+t (-)dt'. In Jll| it was found that a had 
a non-trivial structure both as a function of the re- 
duced dimension of the strategy space d = D/N and 
of the temperature. The second important observable 
is the fraction 4> of "frozen" agents, defined as those 
for which one of the strategies always outperforms the 
other, 4> ee iV _1 ^ 5 (\{%i{t))\ — 1), with the normaliza- 
tion (5(0) = 1. It was introduced in |10| as an order 
parameter for the MG, where it was found that <j){d) 
changed from zero to a finite value at d — d c . 

We now consider the continuous time limit of Eqs. (|l|) 
in such a way as to preserve all the macroscopic features 
of the TMG. To this end we introduce an arbitrary time 
step At. We deal first with the information lit). Let 
us assume that I(t) is a differential random motion in 
the space of strategies, i.e., I{t) — AW(t), with zero 



mean and variance At. Replacing in Eqs. (|l|) we obtain 
Pi(t + At) = Pi (t) - a(t) ■ AW(t) (i ■ AW(t). In the limit 
At — > 0, and using the Kramers-Moyal expansion |p^| , 
we get 



d Pi (t) = -a(t) ■ & dt/D + 0(dt 2 ). 



(2) 



Note that to O(dt) the noise has been eliminated in 
favour of an effective interaction among the agents, and 
the a becomes a 2 = (NDy 1 (a(t) ■ a(t)). 

At T = 0, corresponding to the MG, Eqs. (||) are com- 
pletely deterministic. To first order in dt we have 



dpi(t) 



[hi + 2J Jij z j(t)] dt, 



(3) 



where hi = Ylj &j ' & /ND and Jij 
the volatility reads 



& • £i/ND, while 



a 2 = n + 2 h i + J *i z i (*))' ( 4 ) 



and O ee c3i ■ ujj /ND. In order to check the continu- 
ous time approximation at T = we have simulated Eqs. 
(||) . Results are presented in Fig. |[ We can see that this 
approximation reproduces all the features of the original 
MG. Note that in Eqs. (||) all stochasticity coming from 
the information has dropped out, the only effect being a 
small quantitative deviation in the low d region. 




FIG. 1. Volatility a as a function of the reduced dimen- 
sion d = DJN. Squares ■ correspond to the original dynam- 
ics Eqs. (Ilj); circles O to simulations of Eqs. (til), where an 
Euler algorithm has been used with time step dt = 0.05; dia- 
monds O to the approximation of [11]. In the inset we show 

as a function of d. Average over 100 samples; iV = 100; 
t = to = 10 4 . 



Eqs. (S) can be rewritten as dp/dt 
p ee (pi, . . . ,Pn), similarly for s, and 
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-V S W, where 



(5) 



This is similar to what was done in pi| for the time and 
information averages of Zj. There the value of a was 
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related to the average extrema of TL by assuming that 
the system equilibrated. A good agreement with the nu- 
merics was found in the phase d > d c , but this method 
failed to reproduce the behaviour in the d < d c phase 
(see Fig. [l]). This disagreement was speculated as due 
to the need for terms with higher order time derivatives 
in the continuous time equations. This is clearly wrong, 
since, as we have just shown, Eqs. (||) describe correctly 
the dynamics of the model for all values of d (see Fig. [l]) . 

The phase d < d c of the MG is very sensitive to the ini- 
tial conditions. In Fig. ||we show the results of simulating 
both the original dynamics Eqs. ([[]) and the continuous 
time approximation Eqs. (||) starting from random ini- 
tial conditions |ft(0)| = 0(1). From Fig. ^ see that the 
behaviour of both a and 4> is different from that of Fig. |l| 
in the region d < d c : the system stays in the better-than- 
random phase for all values of d. Again the continuous 
time dynamics is very close to the original discrete one. 
This sensitivity of the results to the initial conditions is 
a clear indication that the system does not equilibrate 
for d < d Cl and raises a question on the existence and 
character of the "phase transition" in the MG . 




FIG. 2. Volatility as a function of d for random initial 
conditions |pi(0)| = 0(1), for the original dynamics Eqs. (^) 
and the continuous time approximation Eqs. Dotted lines 
correspond to zero initial conditions and the approximation 
of [11]. In the inset we show the fraction of frozen agents 
4>. Symbols and details of the simulations are those of the 
previous figure. 

When the temperature is different from zero the TMG 
Eqs. (||) still depend on the stochastic choice of strate- 
gies Ri(t), even at leading order. At each time step, 
R* takes one of the two possible values R\' 2 , defining 
a stochastic jump process. In order to write the corre- 
sponding Master Equation we need to know the tran- 
sition probabilities. The r.h.s. of Eq. (||), which we 
denote Aj, is a normalized sum of N random numbers 
£i • R* (t) , each with mean £j • ujj + £j • £j Sj (t) , and vari- 
ance (£i • £j) 2 [l — Sj(t)]. By the central limit theorem, 
we know that for N large A^ will tend to be normally 
distributed with mean ((A^)) = dH/dsi, and variance 



((A 2 )) = J2j Jij[l - Sj(t)], where ((•)) stands for average 
over realizations of the random process n(t). Moreover, 
A, and Aj^ are correlated, the covariance matrix given 
by 

= J2 J ikJjk[l-sl(t)]. (6) 
k 

Collecting these results, we obtain the transition prob- 
abilities in the large N limit, VF(p'|p) = $(V S H;M), 
where $ corresponds to the normal distribution with 
mean V s 7i and covariance matrix M = {M^}. Note 
that &H/ds l ~ (9(1), and M tj ~ 0(l/N), so that fluctu- 
ations are also of 0(1) and thus are not suppressed when 
N -c oo. 

The R*(t) are chosen independently at each time. If 
we make the natural assumption that in the limit dt —> 
their correlation at different times is a ^-function, the 
Master Equation becomes a Fokker-Planck equation by 
means of Kramers-Moyal expansion |l5| ] 

We therefore conclude that the dynamics of the TMG 
is effectively described by a set of stochastic differential 
equations for the point differences 

dp = -V S H dt + M- dW, (8) 

where W(t) is an A^-dimensional Wiener process, and the 

volatility is given by a 2 = 2(H) + J2i J a - J2i J-n( s f)- 

We have checked by means of extensive numerical sim- 
ulations that Eqs. (|^) give the same results as Eqs. (^), 
up to statistical errors. Figs. || and |] present the results 
from the continuous time dynamics Eqs. (||). For these 
simulations we have chosen for the strategy-use proba- 
bilities the form 7r*' 2 (t) oc exp±/3zi(i) which makes the 
numerics simpler. Similar results can be obtained with 
the form (t) oc exp/3P(i? z 1,2 , t), but a small-p cutoff 
of 0(di) is required to avoid the system getting trapped 
in the p = region. In Fig. || we plot the volatility as a 
function of the temperature, showing that the behaviour 
is the same as the one found in fl3|| : for d < d c , as T is 
increased a first drops to a minimum, and then increases 
towards the random case a — 1; for d > d c , the optimum 
value is the MG one, and a simply grows monotonically 
to 1 . In the inset we give a as a function of d for different 
temperatures. 

Fig. ^ shows how the fraction of frozen agents varies 
as the temperature is increased. For all values of d there 
is a clear jump at T = 0(1) from the MG value to = 1 
]l6[ . Figs. |^ and ^ determine the phase diagram of the 
TMG in the (d, T) plane. It is schematically depicted in 
the inset of Fig. [|. For low d and T the system performs 
worse than random, while for large enough values of T 
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the system becomes random, independently of d. Better 
than random performance is achieved between these two 
regions. It is important to note that, as in the case of the 
MG, the phase structure of the TMG depends strongly on 
the initial conditions. In particular, the low (d, T) part 
of the phase diagram shrinks to zero for finite random 
initial conditions. 
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FIG. 3. Volatility as a function of the temperature 
from the continuous dynamics Eqs. (|^). Inset: volatility 
as a function of d for different values of the temperature 
T = 10~ 3 , 1, 2, 10 0, □, ♦, and A, respectively). A second 
order stochastic Heun algorithm has been used with time step 
dt = 0.02. Average over 20 realizations of the Wiener process 
and 50 samples; N = 100; t = to = 10 4 ; initial conditions 
p(t = 0) = 0. 

In the case where the probabilities are defined as 



1,2/ 



7r^' J (t) oc exp/3P(i? l 1:2 , t), the monotonic increase of a 
to the random value at large T reported in jl3| is due to 
finite waiting times, as pointed out in [jl7|: for t 3> NT 
the volatility stays at the minimum value for any finite 
T> 1. This phenomenon is easily understood from Eqs. 
(||). For large values of T, there is first a transient in 
which Si — tanh j3pi are close to zero, and Eqs. (H) re- 



duce to dpi 



hi dt + J^jk JikJjkdWj 



the point 



differences of all the agents do a randomly biased Brow- 
nian motion, and the system performs as in the random 
case. Eventually, however, p.; become of 0(T) and Si fi- 
nite, and the system effectively behaves as for T ~ 0(1). 
Note that this cannot happen when Sj = tanh/^z^. 

Eqs. (ph are much simpler than the original ones for 
the microscopic description of the TMG. The external 
information has been replaced by an interaction among 
the agents and the random strategy choice has given rise 
to the diffusive term. They describe a dynamics which is 
different from the relaxation of disordered systems usu- 
ally found in physical problems: the random force V s 7i 
can be written as the gradient of a potential function only 
up to a factor, which amounts to a metric in the space of 
p, and the non-trivial diffusion matrix M depends both 
on the variables p and on the quenched disorder of the 
problem. Finding adequate analytic asymptotic solutions 
to this dynamics is the next challenging task. 
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FIG. 4. Fraction of frozen agents (dasa function of T. In- 
set: schematic phase diagram of the TMG in the (d, T) plane. 
Dashed lines indicate crossovers rather than sharp transitions. 
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